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ABSTRACT 

Plugins are commonplace in audio production environments, 
however a common standard has not been developed for the 
web, utilising the Web Audio API. In this study we define 
a standard that can be deployed in web-based production 
environments by defining the plugin structure and host in- 
tegration. The standard facilitates a novel method of cross- 
adaptive processing where features are transmitted between 
plugin instances instead of audio routing, saving on multi- 
ple calculations of features. The project will also enable the 
collection and delivery of semantic information to further 
the field of Intelligent Music Production. 

1. INTRODUCTION 

The Web Audio API {T| defines a cross-browser interface 
for real-time audio processing. The API is supported on 
all major desktop browsers and has led to a wide range 
of applications, including additive synthesisers |2j and full 
production suitesQ Web Audio API plugin standards have 
been proposed before, such as Web Audio API extension 
(WAAX) § or Tunt0 These build audio effects nodes sim- 
ilarly to the Web Audio API’s defined nodes, which are too 
restrictive for a full plugin standard. The Web Audio Mod- 
ules (WAM) |4) define a processor / editor interface, how- 
ever these assume the plugin processor is entirely custom 
DSP code and does not support the use of the streamlined 
audio nodes. 

JSAlj^] (JavaScript Audio Plugin) is a new standard to 
build audio plugins for the web. It defines both the host in- 
terface (PluginFactory) and the plugin structure 
(BasePlugin) with all audio processing performed using 
the web audio API. The standard also defines a novel feature 
sharing method for building auto-/cross- adaptive effects as 
well as linking plugins with the session through semantic 
terms. 


2. ARCHITECTURE 

Audio processing is performed using the web audio API. 
Therefore each plugin instance holds an audio graph, called 
the ‘sub-graph’ . This is private to the plugin and cannot be 
controlled directly unless exposed by design. Each plugin 
instance defines a number of input and output connection 

1 Soundtrap uses the web audio API. available at https ://www. 
soundtrap . com/ 

^ Available at https : //github . com/ Theodeus/ tuna/ 

3 Available at: http : //www . semanticaudio . co . uk./ jsap 


points (which are web audio nodes). Each connection point 
can carry multiple channels depending on the configuration 
of the audio API stream passing through. 

All parameters are defined within the plugin by building 
a custom parameter object. This object supports floating 
point numbers, text, boolean and event (button) style inter- 
face objects as well as ranges. The parameters are stored 
locally to the plugin are are accessed by calling getParam- 
eters() on the plugin instance, returing a JavaScript object 
holding the parameters and their respective values. Like- 
wise the converse setParameters() accepts an object holding 
parameter name / value pairs, facilitating easy manipulation 
of multiple parameters at once. 

Each plugin can also build a custom graphical user inter- 
face (GUI), returning a HTML tree for the host to display. If 
no GUI is generated in the plugin, or the host cannot show 
the desired GUI, the host must still display all the parame- 
ters. This can be styled as the host wishes. 

The PluginFactory is aparent node defining the in- 
terface to all plugin instances and prototypes. The factory 
has a built-in asynchronous loader for downloading, parsing 
and storing prototypes from external JavaScript files. The 
plugin instances are loaded into SubFactories which hold a 
chain of plugins. The SubFactory, on construction, is passed 
two audio nodes defining the start and stop points of the 
chain. New plugin instances are inserted into this chain. 
Plugins can be moved around the chain, deleted or moved 
to other SubFactory instances. 


3. FEATURES AND SEMANTICS 


The PluginFactory can be provided links to data stores al- 
lowing plugins to connect to semantic networks or other 
functions without defining these connections themselves. 
The factory also handles inter-plugin feature sharing. 

Cross-adaptive plugins are defined as plugins whose pa- 
rameters are controlled by another audio channel’s infor- 
mation @ 0 - Early systems used external microphones 
and analog components [7j, a style still used in live en- 
vironments [8j|9|. Other effects perform all-channel com- 


putations 1 10 11| which could be classed as a large auto- 


adaptive effect as no external channels are used. 

For all cross-/auto- adaptive effects, the control signals 
are calculated based on audio features. If multiple plugins 
request the same feature from the same audio stream, tra- 
ditional systems would waste resources re-calculating the 
same feature since the audio would be routed and feature 
extraction handled locally. With the factory, the requested 


Licensed under a Creative Commons Attribution 4.0 International License (CC BY 4.0). 


Proceedings of the 2 nd AES Workshop on Intelligent Music Production, London, UK, 13 September 2016 


plugin calculates the desired features and sends them to the 
factory which dispatches the features to the correct plugin. 
This messaging system also saves handling potentially com- 
plex audio routing paths as well as saving on the complexity 
of internally building and managing a feature extraction. All 
plugin outputs are attached to a JS-Xtract feature extraction 
unit [12J. 

The system supports several ontologies allowing data 
to be collected using the standard and stored in a linked 
database. m show how using plugins with a semantic store 
can benefit audio production, where producers enter terms 
describing the desired sound and the plugin sets the param- 
eters to match. 

The PluginFactory can be fed global information such 
as tempo, audio sample rate, user information and certain 
events. The SubFactories are then fed track specific terms 
such as track name, instrument, group name and audio event 
locations. Most of these can be described using the studio, 
event and timeline ontologies. Each plugin itself is given 
these semantic descriptions enabling it to understand its lo- 
cation. Each plugin also constructs its own semantic in- 
formation either in definition (name, plugin type etc.) or 
through use (paramter movement events, effect transforms 
etc.). 

4. DEPLOYMENT 

The code is in a single JavaScript file available from http : 

/ / www . semanticaudio . co . uk/ jsap/ as well as ex- 
amples and documentation. 

A first use-case of the standard is onlin43 where the 
SAFE plugins (13 ) have been converted into JSAP plug- 
ins. These plugins will be used to extend the SAFE dataset 
through more targeted collection of terms. 

5. CONCLUSION 

This paper introduces the JSAP standard for building intel- 
ligent audio plugins for the web audio API. The standard in- 
troduces the two main components that developers will have 
to use to host and build plugins. The novel feature sharing 
should enable more complex effects to be built along with 
the power of the semantic web to drive the next generation 
of audio effects. 
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